What Gets Retrieved Gets Reinforced: Gender Bias in Agents

Gender bias in AI-powered search is easy to miss and hard to measure. When an LLM mediates between a user and a catalog, bias can enter at two layers: what gets retrieved and how the agent narrates it. No standard tooling exists to measure it in production.

We built a two-layer detector evaluating linguistic patterns in agent responses and stereotypical product selection in retrieval results. The linguistic side worked reasonably well. The retrieval side was a mess: you cannot tell the difference between an agent that is biased and a catalog that already skews gendered. Every pink product flagged as bias was just… the catalog.

Counterfactual testing turned out to be the fix. Swap one name, while everything else remains identical: “Hi, I’m Alice” vs “Hi, I’m Andrew.” If the agent responds differently, that difference is the agent’s doing, not the data’s. We built 400+ test cases across five bias types grounded in Grep-BiasIR, ran them across five LLMs, and found that even well-regarded models fail more often than one would expect.

Whether you are building agentic search or just starting to think about fairness in your pipelines, you will walk away with a concrete testing approach and a clearer picture of where current tools still fall short.

Maria Lungu

Algolia

Maria Lungu is a software engineer at Algolia with eight years of experience across financial services and AI-powered search. Her work has evolved from frontend and UX into fullstack and data engineering, and she currently focuses on building and understanding agentic systems. She is particularly interested in the human side of AI: what these systems get right, what they get wrong, and how to tell the difference.